Multimedia systems integrate multiple sources of digitally represented audio and video for simultaneous, synchronized presentation over an integrated set of output devices. For example, audio information might be stored on magnetic tape or a compact disc (CD), or obtained directly from a microphone coupled to a digital sampler; video information can originate with a videodisc, CD (possibly, although not necessarily, the same CD containing audio information), or an active digital video camera. A multimedia presentation can also include graphical components, text, and still images, all stored digitally and introduced into the presentation at appropriate times. The presentation is conveyed to a viewer over output devices capable of converting the various digital representations into their native form. Multimedia output systems can range, for example, from no more than a computer-driven video monitor and associated sound hardware to integrated high-end stereo and digital video projection systems.
Managing the equipment that permits multimedia presentations to be created from their individual audiovisual components, stored, and later retrieved for playback requires substantial processing power. The multimedia information itself can involve prodigious amounts of digital data even for relatively short presentation sequences, all of which must be handled in a manner that preserves temporal interrelationships. Ordinarily, computational responsibility for the various routing and control aspects of multimedia presentations is shared between a high-level multimedia application program, with which the user interacts to compose, edit, and view the presentation, and a multimedia operating system that performs basic, low-level operations such as data retrieval and output-device designation. This architecture simplifies the creation of multimedia applications by relieving programmers of the need to repeatedly specify routine data-handling operations at the computer system's most basic level; the multimedia operating system acts as a bridge between high-level, user-oriented application tasks and low-level input/output (I/O) and processor commands.
Recognizing the growing importance of multimedia applications, designers of basic computer operating systems have upgraded their systems to include specific facilities for multimedia support. These systems effectively have "built-in" multimedia architectures that programmers can invoke as appropriate. Representative of such systems is the IBM Multimedia Presentation Manager/2.TM. (MMPM/2.TM.) supplied by International Business Machines Corporation (IBM) as an extension to its OS/2.RTM. operating system.
Among the tasks performed by MMPM/2 is capture and formatting of raw digital data from real-time peripheral devices such as cameras and digitizers. Typically, a user records audio and video with a traditional transducer device such as a microphone or video camera, which converts the sound or light-intensity pattern into an analog signal. This signal is processed into a stream of digital values by a dedicated hardware circuit, which repeatedly "samples" the magnitude of the analog signal at a rapid rate. The multimedia operating system captures the resulting data stream and stores it as a data file in a format that preserves the temporal or sequence characteristics of the recorded sounds and scenes.
Ideally, audio and video data, each of which may originate with multiple sources (and at different sampling rates), are ultimately merged into a single integrated data file, simplifying editing and playback considerably. AVI (audio/visual interleave) files, for example, contain multiple streams of different types of data temporally "interleaved" with one another. Unless steps are taken to coordinate the data streams during the interleaving process, however, the final AVI file may not reflect proper synchronization among source components.
Unfortunately, actually implementing the necessary coordination can be quite difficult due to the large amounts of data and high streaming rates that characterize even low-quality multimedia presentations. Hardware solutions, while potentially effective, involve specialized equipment that imposes costs and installation requirements. Software-based solutions have thus far proven largely ineffective. Current interleaving software frequently operates inefficiently, degrading system performance, and may also operate unreliably, producing synchronization of varying quantity when the AVI file is played back.